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(54) Method and apparatus for reading invisibly encoded sound data on an object 



(57) Both a method and apparatus for reading 
sound data invisibly imprinted on the surface of an 
object is provided. In the method, compressed sound 
-data is optically4fr^rinted-ento-the surf ac e o f an o bject- 
as a two-dimensional encodement (1 2). An image of the 
encodement is focused onto an image sensor array (44, 
46) that generates a digital signal representative of the 
image. The digital signal is processed into an expanded 
analog signal representative of the sound represented 
by the encodement. The analog signal is then converted 
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FIG. la 




into sound. The method is implemented by a hand-held 
data reader wand (70. 85) having a lens (72), an image 
sensor, a microprocessor (50) for decoding the output of 

foe4maga sensor array into arvanalog^sound signal r and- 

a sound transducer such as a speaker. The method may 
be used to record and play back a voice commentary on 
not only sheet material objects such as photographs 
and greeting cards, but also three-dimensional articles 
such as clothing (98) and furniture (99, 100). 
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Description 

CROSS REFERENCE TO RELATED APPLICATIONS 

[0001] The application is a continuation-in-part of US. 
patent application Serial No. 08/931.575 ffled Septem- 
ber 16. 1997. 

FIELD OF THE INVENTION 

[0002] This invention generally relates to the reading 
of a sound message invisibly encoded onto an object by 
a hand-held reader wand that optically senses the enco- 
dement and translates it back into sound. While the 
invention finds particular application with respect to the 
reading of sound messages encoded onto photographic 
prints, it may be more generally applied to virtually any 
object 

BACKGROUND OF THE INVENTION 

[0003] Printing of visually observable data on still 
image reflection prints is known. For example, rt is well 
known to optically expose time and date data on photo- 
graphic film in a camera which is the reproduced during 
the photographic printing process in human readable 
form on the print. Typically, this optically reproduced 
data appears within the image area of the print. A disad- 
vantage of this arrangement is that it obscures the 
image area and interferes with pleasurable viewing of 
the print Moreover, it is limited to a small amount of 
data, such as day and date. 

[0004] It is also known to imprint data, including 



of the print. Moving the magnetic strip to the back of the 
print reduces its accessibility and makes it awkward to 
reproduce the sound while viewing the print It requires 
a magnetic reader head employing relative motion 

5 between the head and magnetic strip for signal repro- 
duction. Critical alignment between the reading head 
and data track, as well as maintaining intimate contact 
between the head and magnetic strip are important con- 
siderations that make magnetic signal reproduction an 

io unattractive option for sound reproduction from still 
image prints. Additionally, magnetic recording media 
has a limited life in terms of wear-out and inherent loss 
of magnetically recorded data over time. 
[0006] It is known to provide sound tracks on transmis- 

is sive motion picture film by means of an emulsion coated 
on the film that is sensitive to non-visible light such as 
ultraviolet or infrared. A difficulty with this technique is 
that only small fragments of a complete sound recording 
is placed on each film image frame which would be 

20 unsuitable for use with still image prints in which entire 
sound data files are associated with a single print. 
[0007] There is therefore a need for an arrangement 
of storage and reproduction of sound information and 
other possible data on objects such as still image reflec- 
ts tion prints that avoids the problems and disadvantages 
described above. There is, in particular, a need for an 
arrangement that keeps recorded data intimately asso- 
ciated with a still image reflection print to which it relates 
so that it does not easily become lost. The arrangement 

30 should be able to store enough data to encompass an 
extended amount of sound recorded during picture tak- 
ing and also offer convenience and accessibility for 
reproduction with the image print remaining in an album 
or other mounting means. It would also be desirablelT - 

35 the recordation and playback of the sound data could be 
more universally applied to objects other than photo- 
graphic prints where a sound message or commentary 
would be useful. 

40 SUMMARY OF THE INVENTION 

[0008] Generally speaking, the invention is both a 
method and apparatus for reading invisfcly printed 
sound data on an object. In the method of the invention, 

45 compressed sound data ts optically imprinted onto the 
surface of an object as a two-dimensional encodement 
invisible to the human eye. The encodement is optically 
read by focusing fight emanating from the object onto an 
image sensor array, processing the electrical data gen- 

so erated by the image sensor array in response to the 
focused light image, and then converting the sound data 
into sound. While the sensor array is preferably a 
CMOS-type array, it may alternatively be a CCD. More- 
over, the sensor array may be either a two-dimensional 

55 or one-dimensional array. In the case where the sensor 
is a two-dimensional array, it preferably has a resolution 
capacity of at least .3 mega pixels. If the sensor ts a lin- 
ear sensor array, it preferably has a resolution capacity 



sound-related data, on a still image reflection print in the 
form of machine readable code such as a bar code 
which is observable to the human eye but requires an 
electronic sensor to read and decode the data. An 
example of such an arrangement is disclosed in U.S. 
Patent 5.313.235. A disadvantage of such an arrange- 
ment is that the bar code obscures the image rf printed 
on front and also offers a limited amount of data that is 
not conducive to imprinting sound information that was 
recorded during the picture taking event or for recording 
historical annotations while viewing the print. In the *235 
patent, the imprinted data constitutes essentially a 
memory address pointing to an address location in a 
remote memory device where the actual sound data is 
stored. The actual sound data is not physically tied to 
the print Since the sound information is stored sepa- 
rately from the print, the sound information can become 
disassociated from the print and lost. 
[0005] U.S. Patent 4,095.029 is an example of the use 
of magnetic recording strips on photographic prints for 
storage of sound information directly on the print While 
offering the advantage of keeping the sound information 
physically associated with the print, it also has certain 
disadvantages. It provides a limited storage space and 
uses up available image space when placed on the front 
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of 100 pixels per inch and more preferably 200 pixels 
per inch. Additionally, when a linear sensor array is 
used, the method may also include the step of sweeping 
the focused light emanating from the object across the 
linear array by means of a movable mirror in order to 
obviate the need for a manually-implemented scanning 
step. 

[0009] Preferably, the sound data imprinted on the 
surface of the object is both compressed and at least 
partially redundant so as to maximize the amount and 
duration of sound that can be practically imprinted on 
the object, and to minimize the errors in translating the 
encodement back into sound. The optical and printing 
step may be implemented by infrared, ultraviolet, or flu- 
orescent inks which are normally invisible to the human 
eye. Where the object includes a reflective image, the 
optical imprinting may be incorporated into the image in 
a visually non-perceptible manner. 
[001 0] While the object imprinted upon may be a pho- 
tograph, the object may also encompass virtually any 
type of sheet material where a sound recording would 
be desirable. Examples of such sheet materials include, 
but are not limited to, sheet music, concert leaflets, play- 
ing cards, trick cards, flip cards, the board of a game, 
calendars, any page of a publication, the front pages of 
periodicals such as magazines or newspapers, adver- 
tising brochures and leaflets, sticker prints, and busi- 
ness cards. Applications of the method that are 
particularly beneficial to sight impaired individuals 
include food and beverage labels, and paper money. 
The objects imprinted upon may also include textile arti- 
cles, such as T-shirts or other articles of clothing. 
EinaMy. Jhe ,object_iiTipi:inted_upon_rnay include more_ 
three-dimensional type objects, such as bookbindings, 
book covers, covers for CDs, video discs, audio and 
video cassettes, articles of furniture, and structural 
parts of building (that is, walls, doors, etc.), these last 
applications being of principal benefit to sight impaired 
individuals, or persons working in low-light conditions 
(such as photographic dark rooms, and other low-light 
conditions). Other applications for the method include 
the display structures for photographs, such as picture 
frames, frame glasses or album pages. Finally, the 
method of the invention may advantageously be applied 
to greeting cards, post cards, or the headings of person- 
alized or business-type stationery. 
[001 1 ] The invention also encompasses an apparatus 
for reading the aforementioned two-dimensional enco- 
dement Preferably, the apparatus is a reader wand that 
comprises an image sensor array for generating an 
electronic output signal in response to a focused image 
of the encodement, a lens for focusing an image of the 
encodement onto the image sensor array, a data decod- 
ing unit for decoding the electronic output signal into an 
analog sound signal corresponding to the sound repre- 
sented by the optical encodement and a sound trans- 
ducing means for converting the analog sound signal 
into sound. As was the case in the method of the inven- 



tion, the image sensor may be either a twodimensional 
or one-dimenskxial CMOS or CCD array. When a linear 
sensor array is used, the reader wand preferably 
includes a movably mounted mirror for receiving the 
5 focused image of the encodement from the lens and 
scanning it across the linear array so as to obviate the 
need for a manually implemented scanning operation. 

BRIEF DESCRIPTION OF THE DRAWINGS 

w 

[0012] 

Figures 1a and 1b are illustrations of a still image 
reflection print showing data imprinted thereon in 

15 accordance with an aspect of the invention; 

Figure 2 is a block diagram illustrating conversion of 
an audio signal to a printed code utilizing data com- 
pression, encoding, and printing in accordance with 
an aspect of the invention; 

20 Figure 3 is a block diagram illustrating a system for 
reading and decoding compressed audio data 
recorded on a still image reflection print in accord- 
ance with another aspect of the invention; 
Figure 4a illustrates structure and functional blocks 

25 of a reader wand useful in reading and decoding 
data imprinted on a still image reflection print in 
accordance with the invention; 
Figure 4b illustrates a reader wand employing an 
area array scanner obviating the need for want 

30 motion in scanning encoded data on a still image 
reflection print; 

Figure 4c illustrates an alternative embodiment of a 

reader wand of Fig ure 4a; 

Figure 5 illustrates a swipe motion reader for use in 
35 reading data on a still image reflection print by 
scanning motion of the wand reader across a two- 
dimensional printed encodement on the image 
print and 

Figure 6a, 6b. and 6c illustrate how the invention 
40 may be applied to any one of a number of different 
objects. 

DETAILED DESCRIPTION OF THE INVENTION 

45 [0013] Referring to Figure 1a, a still image reflection 
print 10 is shown on which a two-dimensional array 12 
of encoded variable sound data is invisibly imprinted 
directly on a surface portion 13 of the print entirely 
within the image area 1 1. By Variable data" is means 

so data that varies from print-to-print and contains informa- 
tional subject matter which is associated with the image 
print. Print 10 may be produced utilizing any one of well 
known image printing processes. For example, it may 
be produced by optical exposure onto photographic 

55 emulsions of a photographic print paper of a positive or 
negative film image or of a CRT, LCT. LED, or laser 
image derived from a digital image file. Other examples 
would be a digital image print produced by means of a 
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color inkjet printer or a thermal dye transfer printer. 
Another example would be a print produced by means 
of an electrophotographic process. In Figure 1 b, a sim- 
ilar image print 16 is shown in which the image area 15 
is contained within a non-image border 1 7 and the invis- 
ible two-dimensional data encodement of sound 16 is 
contained entirely within the border 17. It will be appre- 
ciated that the data encodement may alternatively par- 
tially overlap the image area 15. The manner in which 
the reflection print is produced is conventional and does 
not. by itself, represent a part of the invention descrfoed 
herein. 

[0014] In accordance with a particular feature of the 
invention, the encodement areas 12 and 16 comprise a 
two-dimensional array of encoded data imprinted onto 
surface portion 13 with an ink or dye that is invisible to 
the human eye when viewed under normal viewing con- 
ditions. As used herein, the phrase "under normal view- 
ing conditions" means viewing under sunlight or normal 
room illumination such as incandescent or fluorescent 
lights or the like. As shown in Figure 1a the optically 
imprinted data is entirely within the image area of the 
print 10 although it will be appreciated that some of the 
data encodement may extend outside the image area 
into the surrounding border if one exists or may be con- 
tained entirely within the border area as shown in Figure 
1b. Because the optical encodement is invisible to the 
human eye under normal viewing conditions, it is possi- 
ble to have the imprinted, encoded data cover just some 
or even all of the image area without interfering with 
viewing of the image print. The data area may be 
imprinted by thermal printing based on a thermal dye 
transfer process uti lining any well known dye having Rig- 
nificant spectral absorption in the infrared region and 
virtually none in the visible spectrum. Other techniques 
usable would be inkjet printing with an ink having signif- 
icant absorption in the infrared region and none in the 
visible spectrum. Alternatively, the imprinting may be 
accomplished by means of special photographic print 
paper having an infrared absorbing layer exposed with 
the two-dimensional encoded array before, during, or 
after the image printing process. 
[001 51 The code imprinting process may utilize mate- 
rials other than an infrared absorption material. For 
example, a dye or pigment having absorption in the 
ultraviolet region and virtually none in the humanly visi- 
ble spectrum may be used. A fluorescent dye may also 
be used of the type which can be excited to fluoresce in 
a portion of the spectrum that is not visible to the human 
eye. The encoded data preferably comprises a two- 
dimensional bar code, created and imprinted as 
described below, representing sound information 
recorded in association with the image print. The sound 
may be recorded in the camera at the time of picture 
taking and transferred to the print at the time the print is 
produced, i.e., after return of the reflection print to the 
user, and printed thereon by a suitable printer as 
described above thus replacing the conventional notion 



of handwritten notes on the back of the print. 
[0016] Turning now to Figure 2, there is shown a sys- 
tem for encoding audio information into a data format 
which is suitable for imprinting onto an image print as 

s shown in Figure 1. In Figure 2, 20 represents a source 
of audio signal which might be a camera microphone or 
prerecorded audio or audio generated after the image 
print is printed. If the signal from source 20 is in analog 
form it is applied to an A/D converter 22 for conversion 

w to digital form and then applied to an audio compression 
module 24 which includes an algorithm which reduces 
the amount of digital data required to represent the 
audio signal. If the audio signal from source 20 is in dig- 
ital form the A/D unit 22 is bypassed. An example of a 

15 suitable audio compression module is Digital Voice Sys- 
tems. Inc., AMBE-1000 Voice Coder. 
[0017] The compressed audio data is then applied to 
an encoding module 26 where the data is encoded into 
a two-dimensional data array utilizing, for example, the 

20 known format of AIM Standard PDF 417. Software and 
hardware for encoding and rendering the data accord- 
ing to this standard is obtainable from Symbol Technol- 
ogies, Inc. as a part of a LS 4904 S2 Scanner System. 
Another example is an encodement known as 

25 "Datastrip" available from Datastrip Corporation. An 
even more preferred encodement is commercially avail- 
able under the trade name "PaperDisk" from Cobble- 
stone Software. Inc., located in Lexington, 
Massachusetts, USA. "Paper Disk" encodement is pre- 

30 fened due to its robustness; that is, it scheme of provid- 
ing redundant information at different locations within 
the encodement area 12,16. It is also preferred due to 

rts ability_to_be^c^cally_ de<x^ 

skewed, that is, tilted at an angle 15° - 20° from perpen- 

35 dicular with respect to the centerline of the lens used in 
the reader wand. 

[0018] The encoded data array is rendered into a 
printable format by means of module 28 which formats 
the encoded data in a known manner into a format 

40 appropriate for utilization by printer 30. Printer 30 may 
be a thermal dye transfer printer, inkjet printer, electro- 
photographic printer or the like adapted for printing the 
medium described above directly onto an existing 
image print Alternatively, unit 30 may comprise a latent 

45 image exposure apparatus for exposing the com- 
pressed, encoded data via an LCD mask, CRT, LED. or 
laser printer to create a two-dimensional encoded latent 
image array on the special print paper described above 
having the spectrally limited layer formed on the print 

so paper. 

[0019] Having created an image print with superim- 
posed encoded data as shown in either of Figures 1a or 
1b utilizing the encoding and printing apparatus of Fig- 
ure 2, there will now be described an apparatus for 
55 reading, decoding, and playing back to the data with ref- 
erence to the system shown in Figure 3. Print 20, which 
may be an album or frame and bearing a two-dimen- 
sional data array 12 as described above, is "reacT by 
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reader apparatus comprising optical focusing system 40 
having a lens, a spectral bandpass filter 42, and an 
image reader comprising a CCD or CMOS image sen- 
sor 44 and image sensor electronics 46 for converting 
the output of the sensor to a digital signal which is then 
stored in memory unit 48. Trie spectral bandpass filter 
42 is of the type which is transparent to the wavelength 
of light modulated by the two-dimensional data array 
imprinted by the "invisible" material on the image print 
and is opaque to other light reflected from the image 
print. An auxiliary light source 43, having a spectral illu- 
mination characteristics which make the encodement 
visfole to the image sensor 44, may be included to 
enhance the detection of the two-dimensional data 
array. Image processor 50 retrieves the data from mem- 
ory unit 48 and manages the order in which the data is 
decoded and decompressed in decoder 52 and decom- 
pressor 54. The operations in units 52 and 54 are 
essentially the inverse of the compression and encod- 
ing operation described in Figure 2 and are performed 
by the Digital Voice Systems and Symbol Technologies 
software described above. The digital data output from 
decompressor 54 is converted to an analog signal by 
D/A converter 56 and applied to one or more output 
devices such as a speaker 58, or to remote transducer 
units via a cable 60 or transmitter 62. 
[0020] Figure 4a shows a hand-held wand unit 70 that 
incorporates the spectral filter and the circuit and 
processing modules described with respect to Figure 3, 
including a built-in speaker 81. To this end, the wand 
unit 70 comprises a housing 71 containing a focusing 
lens 72, a spectral fitter 73, which may be coated 
directly on the lens 72. and a two-dimensional imag e 
sensor 74 which may be, for example, a video graphics 
array (VGA) sensor having a resolution of 640 x 480 pix- 
els of a type well known in the art or a higher resolution 
16 mega pixel Model KAF-6300 manufactured by the 
Eastman Kodak Company located in Rochester, New 
York. USA. An auxiliary light source 73' similar to light 
source 43 of Figure 3, may be included to enhance 
detection of the data array. In this embodiment, the two- 
dimensional image sensor is used to capture the entire 
twodimensional data array on the image print as shown 
in Figure 4b without the user being required to move the 
wand in a scanning motion over the print. The wand fur- 
ther comprises image sensor electronics 76, memory 
77, processor 78, D/A converter 79, and audio circuit 80 
feeding built-in speaker 81 . Alternatively, in a transmit- 
ting type of wand unit 70, as shown in Figure 4c, audio 
circuit 80 may feed a transmitter unit 83, such as an 
infrared transmitter, for sending the audio information to 
a remote audio playback unit (not shown). Processor 78 
includes the functions of image processor 50, decoder 
52, and decompressor 54 of the reader apparatus of 
Figure 3. 

[0021] An alternative embodiment of the data reader 
wand is shown in Figure 5 in the form of a scanning 
hand-held reader 85 which differs from the wand of Fig- 



ure 4a in that it incorporates a linear sensor array 87 as 
the image sensor. This version of the reader wand 
requires the user to swipe or scan the wand across the 
image print 10 to sense the data on the two-dimensional 

5 data array 1 2. The linear sensor array may be, for exam- 
ple, a NEC Model 3724 tri-linear array sensor having a 
resolution of 200 pixels per inch. To obviate the need for 
the user to swipe or scan the reader 85 across the 
image print 10, the reader 85 may be modified by the 

10 inclusion of a pivotally movable reflective element such 
as a mirror 88 that moves the image focused by the lens 
(not shown) in the reader across the linear sensor array 
87. In either case, the use of a linear sensor 87 is desir- 
able in that it provides higher resolution with a lower 

75 cost sensor array. 

[0022] While the application of the invention has been 
described in detail with respect to photographic prints, it 
may be universally applied to an almost limitless 
number of other objects. For example, as shown in Fig- 

20 ure 6a, the data reader wand 70 may be used to read 
compressed sound data on a surface portion of the 
label of a food product 90, the label of a beverage 91 . or 
even on a piece of paper money 92, these applications 
being of particular interest to the 6ight impaired. More 

25 generally, the invention may be applied to virtually any 
type of sheet material, such as the front cover of a mag- 
azine 93, newspaper, or other periodical, or on sheet 
music 94. In this last application, the compressed sound 
recording could indicate how the melody represented by 

30 the various notes would sound if played by a musical 
instrument, this being of particular interest and benefit 
to a music student. While not specifically shown in any 

of the s everal figures , the invention may be applied to 

other sheet material objects such as business cards, 

35 stationery, the playing boards of board games, any kind 
of playing cards, trick cards, flip cards, the paying 
boards of board games, instruction sheets for the 
assembly of toys or appliances, any page or cover of 
any book or printed publication. Figure 6b illustrates the 

40 application of the invention to more three<limensional 
objects, such as CD covers 95, audio or video cassette 
covers 96. the glass panes 97a or frames 97b that are 
frequently used to mount and display photographs, 
prints, paintings or other images, and even textile arti- 

45 cles such as clothing 98. While not specifically shown in 
Figure 6b. the invention may also be advantageously 
applied to other types of photo or imaging mounts such 
as the clear plastic covers used to mount photos in 
albums. In this last regard, it should be noted that the 

so portion of the clothing invisibly printed with sound data 
could be usefully used as a security tag for the store 
owner, or washing and care instructions for the pur- 
chaser of the article. Finally, as is illustrated in Figure 
6c. the invention may advantageously be applied to 

55 selected surface portions of other three-dimensional 
articles such as chairs 99. tables 100, or other articles 
of furniture, or even structural components of buildings 
such as doors 101 and walls 102. these last appiica- 
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tons again being of particular interest to either the sight 98. Clothing 

impaired or to persons working in low light environ- 99. Chairs 

ments such as photographic dark rooms. 1 00. Tables 

101. Doors 

PARTS LIST s 102. Walls 

[0023] Claims 

10. Still image reflection print 1 . A method of storing and reading sound data, char- 

11. Image area 10 acterized by the steps of : 

12. Two-dimensional array 

13. Surface portion optically imprinting compressed sound data 

15. Image area onto an object as a two-dimensional en cod e- 

16. Invisible two-dimensional data encodemerrt merit (12) invisfole to the human eye; 

17. Non-image border 15 optically reading said encode men t by focusing 
20. Source of audio signal light emanating from said object onto an image 
22. A/D converter sensor array (44, 46); 

24. Audio compression module processing electrical data generated by said 

26. Encoding module image sensor array in response to said focused 

28. Module 20 light into sound data, and 

30. Printer converting said sound data into sound. 

40. Optical focusing system 

42. Spectral bypass filter 2. The method of storing and reading sound data as 

43. Auxiliary light source defined in claim 1, wherein said sensor array is a 

44. Image sensor 25 linear sensor (87), and said optical reading step 
46. Image sensor electronics includes the step of relatively moving said focused 
48. Memory unit light from said object over said linear sensor. 

50. Image processor 

52. Decoder 3. The method of storing and reading sound data as 

54. Decompressor 30 defined in daim 2. wherein said relative movement 

56. A/D converter step is implemented by moving a mirror (88) that 

58. Speaker reflects said focused light from said object across 

60. Cable s aid linear s ensor array. 

62. Transmitter 

70. Hand-held wand unit 35 4. The method of storing and reading sound data as 

71 . Housing defined in claim 1. wherein said object includes an 

72. Focusing lens image, and said compressed sound data is non- 
73. Spectral filter perceptibly incorporated into said image on said 
74. Image sensor object. 

76. Image sensor electronics 40 

77. Memory 5. A hand held data reader wand for reading sound 

78. Processor data imprinted as a two-dimensional optical enco- 

79. D/A converter dement on a surface of an object by means of an 

80. Audio circuit imprinting material invisible to the human eye, the 

81. Speaker 45 wand characterized by: 
83. Transmitter unit 

85. Scanning hand-held reader an image sensor array for generating an elec- 

87. Linear sensor tronic output signal in response to a focused 

88. Mirror image of said encodement; 

90. Food product so a lens (72) for focusing an image of said enco- 

91 . Beverage dement onto said image sensor array; 

92. Paper money data decoding unit for decoding said electronic 

93. Magazine output signal into an analog sound signal corre- 

94. Sheet music sponding to the sound represented by the opti- 

95. CD covers 55 cal encodement, and 

96. Audio/video cassette covers sound transducing means for converting said 
97a. Glass panes analog sound signal into sound. 

97b. Frames 
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6. The hand held data reader wand for reading sound 
data as defined in claim 5, further comprising a 
movable mounted mirror for receiving said focused 
image of said encodemerrt and moving ft across 
said linear array. s 
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